Explaining classifiers by constructing familiar concepts

Abstract Interpreting a large number of neurons in deep learning is difficult. Our proposed ‘CLAssifier-DECoder’ architecture ( ClaDec ) facilitates the understanding output an arbitrary layer or subsets thereof. It uses decoder that transforms incomprehensible representation given to more similar domain human familiar with. In image recognition problem, one can recognize what information (or concepts) maintains by contrasting reconstructed images with those conventional auto-encoder(AE) serving as reference. An extension allows trading comprehensibility and fidelity. We evaluate our approach for classification using convolutional neural networks. show visualizations encodings from classifier capture relevant than AEs. This holds although AEs contain on original input. user study highlights even non-experts identify diverse set concepts contained are irrelevant) classifier. also compare against saliency based methods focus pixel relevance rather concepts. tends highlight input areas though outcomes depend architecture. Code at https://github.com/JohnTailor/ClaDec